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DETAILED ACTION 

- A request for continued examination under 37 CFR 1.114, including the fee set forth in 
37 CFR 1.17(e), was filed in this application after final rejection. Since this application is 
eligible for continued examination under 37 CFR 1.1 14, and the fee set forth in 37 CFR 
1.17(e) has been timely paid, the finality of the previous Office action has been 
withdrawn pursuant to 37 CFR 1.1 14. Applicant's submission filed on 12/1/08 has been 
entered. 

- Claims 1-21 are presented for further examination. 
Claims 1, 7, 8, 20, and 21 are amended. 



Response to Arguments 
Applicant's arguments with respect to claims 1-21 have been considered but are moot in 
view of the new ground(s) of rejection. 



Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

Claims 1-4, 6-10, 16, 20 and 21 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Lundblad et al. hereinafter Lundblad (US 6,906,755 B2) in view of Yap et al 



hereinafter Yap (US 2001/0033343). 



Application/Control Number: 10/509,084 
Art Unit: 2451 



Page 3 



1 . Regarding claim 1 , Lundblad discloses 

A method of synchronizing the delivery of (a) first information (audio, refer to Col 6, Lines 10- 
36) which is to be presented to a user via first output means of a multi-modal interface (b) second 
information (video, refer to Col 6, Lines 10-36) which is to be presented to the user via second 
output means of the multi-modal interface, the method comprising: 

i) estimating total time needed to deliver the first information (a) to the first output means 
or (b) to a store local to the first output means (column 1, lines 55-58); 

ii) estimating total time needed to deliver the second information (a) to the second output 
means or (b) to a store local to the second output means (column 1, lines 58-59); and 

iii) using the estimates obtained in step i) or step ii) to determine whether presentation to 
the user of the first or second information needs to be delayed to achieve a desired synchronism 
of presentation (column 1, lines 59-62); and 

iv) applying any delay determined in step iii) to achieved the desired synchronism of 
presentation (column 1, lines 62-65). 

Although Lundblad disclosed the invention substantially as claimed, Lundblad did not 
explicitly disclosing that the first and second information are coming from two sources. 

Yap, in analogous art, disclosing that the first information and second information are 
coming from two sources (stored separately, refer to par 0116). 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to combine Lundblad and Yap because Yap's teaching of "the first information and 
second information are coming from two sources" would improve Lundblad' s system efficiency 
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by allowing the client to receive multiple data streams to reduce periods of inaccessibility in the 
system. 

2. Regarding claim 2, Lundblad and Yap disclosed the method as claimed in claim 1 . 
Lunblad further discloses wherein the first and second output means are provided by a single 
output device (figure 2, element 108 - TV). 

3. Regarding claim 3, Lundblad and Yap disclosed the method as claimed in claim 1 . 
Lunblad further discloses wherein at least one of the first and second output means is a visual 
display means (figure 2, TV screen for display of video). 

4. Regarding claim 4, Lundblad and Yap disclosed the method as claimed in claim 1 . 
Lunblad further discloses wherein at least one of the first and second output means is an audio 
reproduction means (figure 1, TV speaker for audio). 

5. Regarding claim 6, Lundblad and Yap disclosed the method as claimed in claim 1 . 
Lunblad further discloses wherein the first means is a visual display means (figure 1 , video) and 
the second means is an audio reproduction means (figure 1, audio). 

6. Regarding claim 7, Lundblad discloses 

A method of synchronizing delivery of (a) first information which is to be presented to the user 
via a visual display of a multi-modal interface and (b) second information which is to be 
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presented to the user over a visual or an audio interface of the multi-modal interface, the method 
comprising: 

i) estimating total time needed to deliver the first information ~ to the visual display or 
(b) to a store local to the visual display (column 1, lines 55-58); 

ii) estimating total time needed to deliver the second information (a) to the visual or 
audio interface or (b) to a store local to the visual or audio interface (column 1, lines 58-59); and 

iii) using the estimates obtained in step i) or step ii) to determine whether presentation of 
the first or second information to the user needs to be delayed to achieve a desired synchronism 
of presentation (column 1, lines 59-62); and 

vi) applying delay determined in step iii) to achieved the desired synchronism of 
presentation (column 1, lines 62-65). 

Although Lundblad disclosed the invention substantially as claimed, Lundblad did not 
explicitly disclosing that the first and second information are coming from two sources. 

Yap, in analogous art, disclosing that the first information and second information are 
coming from two sources (stored separately, refer to par 01 16). 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to combine Lundblad and Yap because Yap's teaching of "the first information and 
second information are coming from two sources" would improve Lundblad' s system efficiency 
by allowing the client to receive multiple data streams to reduce periods of inaccessibility in the 
system. 



7. 



Regarding claim 8, Lundblad discloses: 
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A method of synchronizing the delivery (a) of first information which is to be presented to the 
user via a visual display of a multi-modal interface and (b) of second information which is to be 
presented to the user over an audio interface of the multi- modal interface, the method 
comprising: 

i) estimating total time needed to deliver the first information (a) to the visual display or 
(b) to a store local to the visual display (column 1, liens 55-58); 

ii) estimating total time needed to deliver the second information (a) to the audio 
interface (b) or to a store local to the audio interface (column 1, lines 58-59); and 

iii) if the total time estimated in step i) is more than that estimated in step ii) delaying 
presentation of the second information to the user sufficiently to enable the first information to 
be presented to the user before the second information is presented to the user (column 1, lines 
58-62; figure 5, elements 510 and 514). 

Although Lundblad disclosed the invention substantially as claimed, Lundblad did not 
explicitly disclosing that the first and second information are coming from two sources. 

Yap, in analogous art, disclosing that the first information and second information are 
coming from two sources (stored separately, refer to par 01 16). 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to combine Lundblad and Yap because Yap's teaching of "the first information and 
second information are coming from two sources" would improve Lundblad' s system efficiency 
by allowing the client to receive multiple data streams to reduce periods of inaccessibility in the 
system. 
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8. Regarding claim 9, Lundblad and Yap disclosed the method as claimed in claim 7. 
Lunblad further wherein the delivery of the first information in step (i) is controlled by a server 
process, delivery of the first information involving delivery of that information to a client of the 
server process (figure 1, element 102 - content server). 

9. Regarding claim 10, Lundblad and Yap disclosed the method as claimed in claim 7. 
Lunblad further wherein the delivery of the second information in step (ii) is controlled by a 
server process, delivery of the second information involving delivery of that information to a 
client of the server process (figure 1, element 102 - content server). 

1 0. Regarding claim 1 6, Lundblad and Yap disclosed the method as claimed in claim 7. 
Lunblad further wherein the estimate of total time produced in step i) includes a component for 
the time taken to render the first information on the visual display (column 6, lines 8-11, 
"presentation delays"; column 7, lines 56-59). 

1 1 . Regarding claim 20, Lundblad teaches: 

A system of apparatus for the delivery (a) of first information which is to be presented to the user 
via a visual display of a multi-modal interface and (b) of second information which is to be 
presented to the user over a visual or an audio interface of the multi-modal interface, the system 
including processing means configured to: 

estimate total time needed to deliver the first Information to the visual display or to a 
store local to the visual display (column 1, lines 55-58); 
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estimate total time needed to deliver the second information to the visual or audio 
interface or to a store local to the visual or audio interface (column 1, lines 58-59); and 

to use the estimates thereby obtained to determine whether-t-he presentation to the user 
of the first or second information to the user needs to be delayed to achieve a desired 
synchronism of presentation (column 1, lines 59-62), and 

to cause any delay determined to be necessary to be applied to achieve the desired 
synchronism of presentation (column 1, lines 62-65). 

Although Lundblad disclosed the invention substantially as claimed, Lundblad did not 
explicitly disclosing that the first and second information are coming from two sources. 

Yap, in analogous art, disclosing that the first information and second information are 
coming from two sources (stored separately, refer to par 01 16). 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to combine Lundblad and Yap because Yap's teaching of "the first information and 
second information are coming from two sources" would improve Lundblad 's system efficiency 
by allowing the client to receive multiple data streams to reduce periods of inaccessibility in the 
system. 

12. Regarding claim 21, Lundblad teaches: 

A system of apparatus for the delivery (a) of first information which is to be presented to the user 
via first output means of a multi-modal interface and (b) of second information which is to be 
presented to the user via second output means of the multi-modal interface, the system including 
processing means configured to: 
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estimate the total time needed to deliver the first information to the first output means or 
to a store local to the first output means (column 1, lines 55-58); 

estimate the total time needed to deliver the second information to second output means 
or to a store local to the second output means (column 1, lines 58-59); and 

to use the estimates thereby obtained to determine whether presentation of the first or 
second information to the user needs to be delayed to achieve a desired synchronism of 
presentation (column 1, lines 59-62); and 

to cause any delay determined to be necessary to be applied to achieve the desired 
synchronism of presentation (column 1 , lines 62-65). 

Although Lundblad disclosed the invention substantially as claimed, Lundblad did not 
explicitly disclosing that the first and second information are coming from two sources. 

Yap, in analogous art, disclosing that the first information and second information are 
coming from two sources (stored separately, refer to par 0116). 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to combine Lundblad and Yap because Yap's teaching of "the first information and 
second information are coming from two sources" would improve Lundblad' s system efficiency 
by allowing the client to receive multiple data streams to reduce periods of inaccessibility in the 
system. 

Claim 5 is rejected under 35 U.S.C. 103(a) as being unpatentable over Lundblad et al. 
hereinafter Lundblad (US 6,906,755 B2) in view of Yap et al hereinafter Yap (US 
2001/0033343) in further view of Dutta et al hereinafter Dutta (US 6,961,458 B2). 
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13. Referring to Claim 5, Lundblad and Yap disclosed the method as claimed in claim 1 . 
Lundblad and Yap did not explicitly disclose that the output means are tactile 

reproduction means. 

Dutta teaches a data output mean for visually impaired user. Column 6, lines 1-3 teach 
the output device that generates tactile output like Braille. Dutta further provide the advantage of 
using tactile to communicate information to the visually impaired user. 

It would have been obvious to one of ordinary skill in the art, having the teaching of 
Lundblad, Yap and Dutta before them at the time the invention was made to modify the system 
of Charlton et al. to include tactile output means as taught by Dutta 

One of ordinary skill in the art would have been motivated to make this modification in 
order to allow a visually impaired user to receive data in a communication system. Dutta teaches 
that transmitting and receiving information in a communication system has become culture 
fixture and that visually impaired user relies on tools like tactile to help them be part of that 
culture. 

Claims 11-15 and 17-19 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Lundblad et al. hereinafter Lundblad (US 6,906,755 B2) in view of Yap et al hereinafter Yap 
(US 2001/0033343) in further view of Grove et al. hereinafter Grove (US 6,820,133). 

14. Regarding claims 1 1 and 12, Lundblad and Yap disclosed the method as claimed in claim 
7. 
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Lundblad teaches the determination of processing delay includes presentation delay. 
Lundblad further teach utilizing the measured latency to estimate the communication time 
between two devices (column 18, lines 53-62). 

Lundblad and Yap did not explicitly disclose latency of a communication channel. 

Grove teaches measuring of latency of a data transmission from one node to another 
(Column 15 lines 27-37). 

It would have been obvious to one of ordinary skill in the art, having the teachings of 
Lundblad, Yap and Grove before them at the time the invention was made to modify the system 
and method of Lundblad and Yap to use measured latency for estimation of data transmission 
time as taught by Grove because Lundblad suggests the utilization of time related to transmission 
of data in the presentation delay. 

One of ordinary skill in the art would have been motivated to make this modification in 
order to better estimate the processing delay needed. 

1 5. Regarding claim 1 3, Lundblad and Yap disclosed the method as claimed in claim 1 1 . 
Both Lundblad and Yap did not explicitly disclosing measurement of latency involves a server 
process sending a communication to an associated client to elicit a response therefrom, the 
measurement of latency being derived from the duration of the interval between the sending of 
the communication and the receipt of the response. 

Grove further teaches that the measurement of latency involves a server process sending 
a communication to an associated client to elicit a response therefrom, the measurement of 
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latency being derived from the duration of the interval between the sending of the 
communication and the receipt of the response (column 15, lines 29-32). 

It would have been obvious to one of ordinary skill in the art, having the teachings of 
Lundblad, Yap and Grove before them at the time the invention was made to modify the system 
and method of Lundblad and Yap to use measured latency for estimation of data transmission 
time as taught by Grove because Lundblad suggests the utilization of time related to transmission 
of data in the presentation delay. 

One of ordinary skill in the art would have been motivated to make this modification in 
order to better estimate the processing delay needed. 

16. Regarding claims 14 and 15, Lundblad and Yap disclosed the method as claimed in claim 
7. 

Lundblad further teach that knowledge of the quantity of video (first information) and 
audio (second information) are used to estimate the processing delay (column 5, lines 12-14 and 
column 6, lines 45-62). 

Lundblad and Yap did not explicitly disclose the bandwidth information of the 
communication channel is used to calculate the time required to transmit the information and 
subsequently used in the estimation of the processing delay. 

Grove teaches using bandwidth figures to estimate the communication time between two 
devices (column 18, lines 53-62). 

It would have been obvious to one of ordinary skill in the art, having the teachings of 
Lundblad, Yap and Grove before them at the time the invention was made to modify the system 



Application/Control Number: 10/509,084 Page 13 

Art Unit: 2451 

and method of Lundblad and Yap to use bandwidth information for estimation of the data 
transmission time as taught by Grove. 

One of ordinary skill in the art would have been motivated to make this modification in 
order to better estimate the processing delay needed. 

17. Claim 17 is rejected on the same basis as claims 11-12 and 14-15. See the discussions 
regarding claims 11-12 and 14-15 above for details of this disclosure. 

18. Claim 18 is rejected on the same basis as claims 1 1 and 12. See the discussions regarding 
claims 1 1 and 12 above for details of this disclosure. 

19. Claim 19 is rejected on the same basis as claims 14 and 15. See the discussions regarding 
claims 14 and 15 above for details of this disclosure. 

Conclusion 

Examiner's Notes: Examiner has cited particular columns and line numbers in the 
references applied to the claims above for the convenience of the applicant. Although the 
specified citations are representative of the teachings of the art and are applied to specific 
limitations within the individual claim, other passages and figures may apply as well. It is 
respectfully requested from the applicant in preparing responses, to fully consider the references 
in entirety as potentially teaching all or part of the claimed invention, as well as the context of 
the passage as taught by the prior art or disclosed by the Examiner. In the case of amending the 
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claimed invention, Applicant is respectfully requested to indicate the portion(s) of the 
specification which dictate(s) the structure relied on for proper interpretation and also to verify 
and ascertain the metes and bounds of the claimed invention. 

A shortened statutory period for reply to this Office action is set to expire THREE 
MONTHS from the mailing date of this action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Karen C. Tang whose telephone number is (571)272-3 116. The 
examiner can normally be reached on M-F 7-3. 

If attempts to reach the examiner by telephone arc unsuccessful, the examiner's 
supervisor, John Follansbee can be reached on (571)272-3964. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 
Information regarding the status of an application may be obtained from the Patent Application 
Information Retrieval (PAIR) system. Status information for published applications may be 
obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 



/K. C. TJ 

Examiner, Art Unit 245 1 
/Larry D Donaghue/ 
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